Impact of noise and damage on collective dynamics of scale-free neuronal networks 
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We study the role of scale-free structure and noise in collective dynamics of neuronal networks. For 
this purpose, we simulate and study analytically a cortical circuit model with stochastic neurons. 
We compare collective neuronal activity of networks with different topologies: classical random 
graphs and scale-free networks. We show that, in scale-free networks with divergent second moment 
of degree distribution, an influence of noise on neuronal activity is strongly enhanced in comparison 
with networks with a finite second moment. A very small noise level can stimulate spontaneous 
activity of a finite fraction of neurons and sustained network oscillations. We demonstrate tolerance 
of collective dynamics of the scale-free networks to random damage in a broad range of the number 
of randomly removed excitatory and inhibitory neurons. A random removal of neurons leads to 
gradual decrease of frequency of network oscillations similar to the slowing-down of the alpha rhythm 
in Alzheimer's disease. However, the networks are vulnerable to targeted attacks. A removal of a 
few excitatory or inhibitory hubs can impair sustained network oscillations. 

PACS numbers: 02.50.Ey, 64.60.aq, 87.18.Sn, 87.19.1n, 87.19.1c, 87.19.1j 



I. INTRODUCTION 

Recent investigations have revealed that, on the func- 
tional level, brain has a complex network structure with 
small- world properties and scale-free architecture [H-Q- 
This kind of structure has been found in many other real 
physical, biological, and social systems where it plays an 
important role in dynamics and function of the systems 
It has been found that scale- free networks de- 
scribing a number of complex systems, such as the World 
Wide Web, Internet, social networks or a cell, demon- 
strate error tolerance and attack vulnerability [l3l - [l5j |. 
Networks with divergent second moment of degree dis- 
tribution, such as scale-free networks with degree dis- 
tribution P(q) oc q ' when 2 < 7 < 3, are more ro- 
bust against random damage in comparison to networks 
with a finite second moment of degree distribution, such 
as scale-free networks with degree exponent 7 > 3 or 
classical random graphs. It is practically impossible to 
damage the giant connected component in an infinite un- 
corrected network with divergent second moment of de- 
gree distribution. This kind of network is ultraresilient 
against random damage or failures [HI, • In contrast 
to random damage, the removal of highly connected hubs 
from a scale-free network effectively destroys the con- 
nectivity [H, [H, [l(| . A close relationship of structural 
and functional neuronal networks and the importance of 
topological considerations for the function of the brain 
are discussed in reviews 0, [jj [TtJ • Theoretical inves- 
tigations of neuronal networks revealed that scale-free 
structure may play an important role in synchronous 
neuronal activity [IH, , neuronal avalanches [2(| , and 
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neurological diseases [2l| . An important feature of scale- 
free networks is a large number of hubs in comparison 
with classical random graphs with the same mean de- 
gree. Hubs may play an important role in connectivity, 
promoting hyperexcitability after brain injury 12 ill , pro- 
viding mechanism for orchestrating synchrony [22j or in- 
tegrating multisensory information |23j . Recently, a " rich 
club" of densely interconnected hub regions was found in 
the human brain [24], [2|| similar to " rich club" found in 
other real complex systems [26l . |27| . In the brain, this 
rich club forms a central backbone for global brain com- 
munication [24], H5[. Connectivity plays a critical role 
in mediating cognitive function. Neurodegenerative dis- 
eases target specific and functionally connected neuronal 
networks @,[2^]. Alzheimer's disease is characterized by 
the loss of neurons and synapses in the cerebral cortex 
and certain subcortical regions. This results in a slowing 
down of the alpha rhythm up to disappearance of this 
oscillation in the severe stages [30| . At the present time, 
understanding of a role of the scale-free structure and 
hubs in brain dynamics and mechanisms of the impair- 
ment of brain functions caused by damage of neuronal 
networks is elusive. 

Besides the fact that the brain has a heterogeneous 
structure of complex networks, brain is noisy. Usually, it 
is assumed that increase of noise level leads to increasing 
disorder. However, under certain conditions, noise can 
play a constructive role, leading to stochastic resonance 
and enhancement of weak signals [3l|, [32| or generating 
coherent behavior 33 35]. Various sources of noise were 
identified in the brain such as intracellular and synap- 
tic noise and many others [3£b38j. A role of noise in 
brain activity was already discussed in a lar ge body of 
literature in neuroscience, see recent reviews [37H39j | . In 
the brain, noise stimulates spontaneous neuronal activity, 
fluctuation phenomena, and stochastic effects 38, 40]. It 
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strongly influences both integrative properties of individ- 
ual neurons [41-46] and collective neuronal activity (47i — 
IHol ]. Taking into account the fact that noise affects brain 
dynamics, it is important to understand the following: 
Does complex network architecture of neuronal networks 
play a role in the impact of noise on the brain activity? 

In the present paper, we study the role of the scale- 
free architecture and noise in activity of neuronal net- 
works. For this purpose, we use a cortical circuit model 
[5lj in which neurons form a strongly heterogeneous net- 
work with scale-free structure. In the considered net- 
works, there are many excitatory and inhibitory hubs 
that are mutually and densely interconnected. By use 
of numerical simulations and analytical considerations of 
the cortical model, we show that noise strongly enhances 
spontaneous activity of neuronal networks with scale- free 
structure. In the case of scale-free networks with diver- 
gent second moment of degree distributions of synaptic 
connections, even a weak noise can stimulate spontaneous 
asynchronous activity of a finite fraction of neurons and 
sustained network oscillations. Furthermore, we show er- 
ror tolerance of scale-free neuronal networks to random 
damage in a wide range of the number of randomly re- 
moved excitatory or inhibitory neurons. Only damage of 
a substantial part of the network (about 56% of excita- 
tory neurons or about 25% of inhibitory neurons for the 
networks studied in the paper) impairs synchronization 
between neurons and suppresses sustained network oscil- 
lations. We demonstrate that excitatory and inhibitory 
hubs play a crucial role in orchestrating neuronal activity. 
Targeted attack on hubs effectively impacts collective be- 
havior of neuronal networks. The removal of only a few 
hubs can produce an effect as strong as random removal 
of a finite fraction of neurons. In particular, we find that 
removal of even a small number of hubs decreases the fre- 
quency of network oscillations and ultimately suppresses 
sustained network oscillations. 



II. CORTICAL CIRCUIT MODEL 

In the present paper, we study a cortical circuit model 
of neuronal networks c omp osed of stochastic excitatory 
and inhibitory neurons [5l| . The advantage of this model 
is that it includes biologically important constituent ele- 
ments of real neuronal networks, it does not need time- 
consuming simulations, it takes explicitly into account 
network heterogeneity, and it can be solved analytically. 
The analytical consideration enables us to understand 
mechanisms and origin of collective phenomena and the 
role of finite size effects in dynamics of neuronal net- 
works. Our approach is based on assumption that there 
are universal phenomena in collective dynamics of neural 
networks that are independent on individual dynamics of 
neurons and depend on topological structure of the net- 
work and interaction between neurons. Investigation of 
these universal phenomena is the main aim our paper. 

First, in contrast to models with integrate-and-fire 
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or another kind of individual dynamics 
we consider stochastic neurons like those of 
In this approach, a response of neurons on 
input is stochastic. A sufficiently large input activates a 
neuron with a certain rate as in the Hopfield model [57| . 

Second, we assume that excitatory and inhibitory neu- 
rons are tonic. They fire spikes with a constant frequency 
v for any input larger than a threshold. In this respect, 
neurons behave like those of McCulloch and Pitts [58| . 
The firing frequency v is the same for both excitatory 
and inhibitory neurons. Spikes mediate interactions be- 
tween neurons. Total input from presynaptic neurons 
consists of spikes from excitatory neurons that activate a 
postsynaptic neuron and spikes from inhibitory neurons 
that suppress activity of the postsynaptic neuron. The 
total input Vj(t) at neuron j at time t can be written 
as sum of contributions from excitatory and inhibitory 
neurons arriving during the time interval [i — r, t] where 
t is the integration time: 
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(i) 



where Jp.j is the efficacy of the synaptic connection be- 
tween pre- and postsynaptic neurons t and j, respec- 
tively. In the case vt > 1, we assume that the number 
ni(t) of spikes generated by active neuron I during time 
r is given by ne(t) = vt. If vt < 1, then npjt) = 1 
with the probability vt, otherwise, ni(t) = [511 ] . In 
the present paper, for simplicity, we will consider the 
case vt > 1. The case vt < 1 leads to qualitatively 
similar results 15 if ] - Inactive neurons give no contribu- 



tion to Vj(t). Furthermore, we assume that all efficacies 
of synaptic connections of excitatory neurons to post- 
synaptic excitatory and inhibitory neurons are uniform 
and positive, Jgj = J e . Efficacies of synaptic connections 
of inhibitory neurons to postsynaptic excitatory and in- 
hibitory neurons are also uniform but negative and equal 
to J{. 

Third, in the considered model, we take into account 
noise. The noise level is characterized by rates f e and fi 
that determine the probabilities r/ e and r/j that an exci- 
tatory or inhibitory neuron, respectively, is activated by 
noise during time interval r. In our model, this stochas- 
tic process of activation of neurons represents action of 
synaptic noise (see Appendix [Cj) . 



A. Static model for scale-free neuronal networks 

In order to study scale- free neuronal networks we use 
the static model (i 1 6.'5 that was developed for random 
uncorrelated scale-free complex networks. We generalize 
the model and build scale-free directed networks with two 
populations of neurons. The model allows us to build a 
strongly heterogeneous network with both excitatory and 
inhibitory hubs. These hubs are mutually and densely 
interconnected. 
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Let us consider a network composed by N e excitatory 
and Ni inhibitory neurons. They are numerated by in- 
dices i = 1,2, . . . , N e and j = 1,2, ... , Ni, respectively. 
The fraction of excitatory neurons is g e = N e /N and 
gi = 1 — g e is the fraction of inhibitory neurons where 
N = N e + Ni is the total number of neurons. The weight 

W a (j) = ^ ^ - (2) 

is assigned to each neuron j = 1 , . . . , iV a in population 
a — e,i. The weights are normalized, i.e., Ylj=i w a(j) — 
1. We consider the case A e = A, = A with A G [0,1). 
These weights are used to construct a directed network. 
The probability to have a directed connection from neu- 
ron £ in population a to a neuron j in population b is 

Pa,e-,b,j = Ng a K ab g b w a (£)w b (j) . (3) 

According to this probability and the weight Eq. ([2]), a 
neuron in population b with a small index j has a larger 
probability to have a connection with neurons in popula- 
tions a in comparison with a neuron with a larger index 
j. Therefore, neurons with a small index j have a large 
probability to become hubs. Neurons in population b 
have in average C ab presynaptic neurons in population a, 
where 

1 N b N a 

Cab = JT 22 zJPa^iM = 5a^ab- (4) 
b 3=1 f=l 

An excitatory neuron has in average C ee of presynaptic 
excitatory and Ci e presynaptic inhibitory neurons. An 
inhibitory neuron has in average C e i of presynaptic ex- 
citatory and Cu presynaptic inhibitory neurons. This 
determines the role of the matrix K ab . 



The in-degree distribution in a scale-free network built 
by use of the probability Eq. §3§ is shown in Fig. [1] One 
can see that the degree distribution is a power law at a 
sufficiently large degree k , 

P(A;)~Ar 7 , (5) 

with the degree exponent 

7 = 1 + ~ • (6) 

Using this model, one builds a network with 7 > 2. Note 
that, in the static model, in- and out-degrees are corre- 
lated because the weight Eq. ^ determines the proba- 
bility Eq. ([3]) of both pre- and postsynaptic connections. 
When building a network, we excluded multiple and self- 
links. In the case 7 < 3, the exclusion of multiple links 
causes degree- degree correlations, namely, disassortative 
mixing [6l|-|63j. 

B. Erdos-Renyi networks 

If, in Eq. ([3]), we suppose that the weights w e (j) = 
1/N e and Wi(j) = then we obtain the probabil- 

ity p a ,£;b,j — K ab /N that does not depend on the in- 
dices £ and j. Constructing a network with this proba- 
bility, we obtain a sparsely connected network with the 
same mean numbers of connections between excitatory 
and inhibitory neurons as in the corresponding scale-free 
network. The sparsely connected network has the Pois- 
son degree distribution as the classical random or Erdos- 
Renyi networks. Comparing dynamics of the static model 
and the corresponding randomly connected network, one 
can study a role of structure in network dynamics. 
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FIG. 1. Degree distributions of presynaptic connections of 
excitatory and inhibitory neurons in the static model Eq. $Jfy. 
The degree distributions of excitatory-excitatory, excitatory- 
inhibitory, inhibitory-excitatory, and inhibitory-inhibitory di- 
rected synaptic connections are obtained by averaging over 25 
network realizations for the following parameters: TV = 10000; 
gi = 0.2; 7 = 2.5; K ee = K u = 150; K el = K ie = 100. 



C. Rules of stochastic dynamics. Algorithm 

In the cortical model, sporadic inputs to neurons gener- 
ate neuronal activity. Dynamics of neuronal populations 
is determined by the following rules: 

(i) During the integration time r, inactive excitatory 
and inhibitory neurons are activated by noise with 
probabilities r/ e and r/j, respectively. 

(ii) An excitatory or inhibitory neuron is activated with 
probability rfi e or 77^, respectively, by an input 
V(t), if V(t) is larger then a threshold V t h- 

(iii) Excitatory and inhibitory neurons stop firing with 
probabilities r[i e and r/ij, respectively, if input 
V(t) becomes smaller than Vth- 

For simplicity, we assume that the threshold Vth is the 
same for all neurons. 

In the case vr > 1, the input Eq. (JXJ) takes a form, 
V(t) = VTj e n + vrJiTri, where n and m are the number 
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of active presynaptic excitatory and inhibitory neurons 
at time t, respectively. It is convenient to introduce a 
dimensionless activation threshold O, il = V t h/{ J e vr). 
The activation condition V(t) > Vth takes a form, 



\Ji/ J e \m > J7. 



(7) 



Below we use the unit J e = 1. fHs of the order of 15-30 
in living neuronal networks [63 - l66| and about 30 — 400 
in the brain. 

We assume that l/p e and are of the order of the 
first-spike latencies of excitatory and inhibitory neurons, 
respectively. First-spike latency is defined as the time 
from the onset of stimulus to the time of occurrence of the 



first- response spike [67(. It is about 6-7 ms in fast-spike 
interneurons in neocortex |68j |. The first spike latency is 
ranged from 25 to 49 ms in CA3 hippocampal pyramidal 
(excitatory) neurons [69fl and 20 -128 ms in inhibitory 
cerebellar stellate cells |70| . In our model, the ratio 
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(8) 



is an important parameter that determines collective dy- 
namics of neurons. According to the experimental data, 
a may be both larger and smaller than 1. 

In simulations, we apply the following algorithm. We 
divide time t into intervals of width At — t. At each time 
step, an inactive excitatory or inhibitory neuron may be 
activated by noise with probability Atf e or Atfi, respec- 
tively. Furthermore, at each time step, for each neuron, 
we calculate the input Eq. (fTJ, taking into account that, 
in the case tv > 1, each active presynaptic neuron con- 
tributes with tv spikes. In the case tv < 1, an active 
presynaptic neuron contributes with a spike with proba- 
bility tv. Then, with the probability Atp a , a = e,i, we 
update the states of all neurons using the stochastic rules 
formulated above. In our simulations, we only consider 
the case tv > 1. Throughout this paper we use l//i e = 1 
as time unit. 
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FIG. 2. Asynchronous behavior and sustained network os- 
cillations in the cortical model with scale-free structure, (a) 
Periodic change of noise level F leads to periodic change of 
activity p e of excitatory neurons: a low neuronal activity at 
a low noise level and sustained network oscillations at a high 
noise level. Inhibitory neurons (not shown) demonstrate sim- 
ilar behavior, (b) Raster plot displays temporal behavior of 
individual excitatory neurons with index 1, ...,4000. At a 
low noise level, the neuronal activities are weak and asyn- 
chronous. At a high noise level, the neurons are synchronized 
and sustained network oscillations appear. Note that neu- 
rons with a small index are hubs and have stronger activity 
in comparison to neurons having a large index and, therefore, 
a smaller number of synaptic connections. Other parameters: 
N = 5000, gi = 0.2, 7 = 2.5, K ab = 400, Ji = -3.2, fi = 60, 
a = 0.1, At = 0.1. 



D. Rate equations for neuronal activity 

Let us study dynamics of neuronal networks with scale- 
free structure of the static model. We simulated the 
model using the algorithm described above in Sec. Ill CI 
The raster plot in Fig. [2] represents activity of neurons 
in the case when the noise level is periodically changed. 
One can see that if the noise level is low, neurons show 
asynchronous behavior while at a high noise level their 
activities are correlated and form sustained network os- 
cillations. In order to understand origin of collective 
behavior and mechanism of synchronization in neuronal 
networks, we develop a theoretical approach to dynamics 
of the cortical model. 

Let us introduce the probability p a (£,t) that at time t 
neuron £ in population a is active. We define the activity 



p a (t) of population a as follows: 

a e=i 

The activity p a (t) represents neuronal activity in EEG 
measurements. We also introduce a weighted activity 
p a (t) of population a, 

p a (t) = ^w a (£)p a (£,t). (10) 
l=\ 

Based on the rules of the stochastic dynamics in Sec. Ill CI 
and the method developed in Ref. [5l|, one derives rate 
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equations for the weighted activities of excitatory and 
inhibitory populations (see Appendix [A]) : 

Pe = fe- "epe + /i e *e(Pe,A), 

Pi = ft - ViPi + P>$i(pe,Pi), (11) 

where p = dp/dt, v a = fa + Pa, and 

^a{pe,Pi) = Y^Wa{j)^ a ^{p e ,Pi) . (12) 
J=l 

The function j(p e , pi) is the probability that, at given 
weighted activities p e and pi, an input at neuron j in 
population a is at least the threshold f2, 
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^ a ,j{pe,Pi) = Pn(peC ea (j))P m {piCia(j)) 
n— O m— 

G(n- \Ji\m-fl). 



(13) 



The product P„(peC ea (j))P m (PiC ia (j)) is the probability 
that, at given weighted activities p e and pi, neuron j has 
n active presynaptic excitatory and m active presynaptic 
inhibitory neurons. The Heaviside step function 0(n — 
|Ji|m — ft) selects events when the input at a neuron 
j is at least the threshold ft, Eq. ([7]). The function 
Pn(x) = x n e x jn\ is the Poissonian distribution function. 
Ceb(j) and C'ib(j) are the averaged number of excitatory 
and inhibitory presynaptic connections of neuron j in 
population b, 

No. 



Solving Eqs. (fTTj) with respect to p e (t) and (t) , then 
one finds the activities p e (t) and pi{t), using equations 



Pe = fe - V e p e + ^ e $ e ( j 5 e , pi 
Pi = ft - V%Pi + Pt$i(pe, Pi), 



where 
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a(Pe,Pt) = J^^2^a,j(p, 



(15) 



(16) 



(see Appendix [AJ . Rate equations (jTTJ) are asymptoti- 
cally exact in the limit of large mean degree and large 
size N ^> 1. Moreover, it is assumed that the integration 
time t is much smaller than the relaxation time or the 
period of sustained network oscillations. 

It is convenient to introduce a dimensionless parame- 
ter, 



F a = 



fa 



fa + Pa 



(17) 



for a = e,i. F a characterizes the activation rate f a of 
neurons stimulated by noise with respect to the rate p a - 



lla 



lib 




FIG. 3. Schematic representation of the a— F plane of the cor- 
tical model. (I) A region with low activity of neurons, asyn- 
chronous behavior, and exponential relaxation to a steady 
state; (II) A region with high activity of neurons. The dotted 
line divides region II into two subregions: region Ha with ex- 
ponential relaxation and region lib with damped network os- 
cillations. (Ill) A region with sustained network oscillations. 
The vertical dashed line is the critical line of a first-order 
phase transition from a state with low neuronal activity to a 
state with high activity when a > a t . Above the critical noise 
level only the state with a high neuronal activity is stable. 
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FIG. 4. Sustained network oscillations of activity of excita- 
tory population in the cortical model from a numerical solu- 
tion of Eq. (solid line) and our simulations (dotted line) 
for parameters: N = 10000, gt = 0.2, 7 = 2.5, K ab = 75, 
F = 0.2, Jt = -3.5, ft = 10, a = 0.1, At = 0.1. 



If F -C 1, it corresponds to a small noise level, i.e., the 
probability of activation by noise is small (see Appendix 
IC|l . For simplicity, we consider the case F e — Fi = F. 

Equations (jlip are non-linear and their explicit solu- 
tion is unknown. We solved them numerically. Analyzing 
a linear response of the activities on small perturbations 
of the noise level, we found that the phase diagram of 
the cortical model with scale-free structure is qualita- 
tively similar to the phase diagram found in Ref. [5l[ 
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for sparsely connected networks with Poisson degree dis- 
tribution. In Fig. [31 there are three regions: region I 
with low activity and asynchronous behavior of neurons 
and exponential relaxation to a steady state; region II 
with high activity of neurons and either exponential re- 
laxation or relaxation in the form of damped oscillations 
to a steady state; region III with sustained network os- 
cillations. 

In Fig. 3] we compare between a numerical solution of 
Eqs. (fTTj) and simulations of the cortical model of size 
N = 10000 with degree exponent 7 = 2.5. One can see 
that shape and frequency of sustained network oscilla- 
tions calculated from Eqs. (ITT]) are in good agreement 
with simulations. This result shows that Eqs. (|TTj) actu- 
ally describe correctly the dynamics of the cortical model. 



III. NOISE-INDUCED SPONTANEOUS 
ACTIVITY AND SUSTAINED OSCILLATIONS 

In Sec. U we have noted that noise plays an impor- 
tant role in activity of neuronal networks. In this section 
we will show that the level of noise necessary to sustain 
activity of a finite fraction of neurons depends on the 
network structure. In scale-free networks with fat tailed 
degree distribution, it can be very small. 



A. Spontaneous activity in scale- free networks 

Even in the absence of any stimulus, neuronal networks 
demonstrate spontaneous activity that may be generated 
by noise. Let us study how spontaneous activity depends 
on structure of neuronal networks. For this purpose, we 
consider the static model of scale-free neuronal networks 
from Sec. Ill Al First we consider the case when a is larger 
than a certain value at above which there are no sus- 
tained network oscillations (at is the a coordinate of the 
top point of the region III in Fig. [3]). In this case, with in- 
creasing the noise level F the activity of the neuronal net- 
work undergoes a discontinuous transition with a jump in 
excitatory and inhibitory activity at a critical noise level 
F c , see Fig. [5] In the schematic phase diagram in Fig. [31 
the critical noise level F c determines the line of the first- 
order phase transition. At a low noise level F < F c , the 
activities are very small because neurons are activated 
asynchronously and rarely by noise. When F approaches 
F c , one observes avalanches stimulated by interaction be- 
tween neurons [o"l| . Avalanches in real neuronal networks 
have been revealed by Beggs and Plenz [7l|, see also re- 
cent reviews [72H74j . A detailed discussion of avalanches 
is out of the scope of the present paper. 

Another interesting phenomenon is hysteresis. Start- 
ing from a weak noise, at first we increased the noise level 
F from zero to a value above the critical point F c and af- 
terwards decreased it again to a value below F c . In Fig.[S] 
one can see that the network activity remains as high as 
it was above F r . In our simulations, we found that the 



neuronal network stays in this metastable active state for 
a long time because the activity is supported by spiking 
neurons. A similar phenomenon was found in a model of 
mammalian thalamocortical systems in Rcf. where 
spiking dynamics of each neuron was simulated by use 
of Izhikevich model [52J . The authors [50( observed that 
the neuronal activity remained high even after turn-off 
the spontaneous synaptic release that was a driving force 
of initial activity. 
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FIG. 5. Excitatory activity p e versus the noise level F at dif- 
ferent fractions gi of inhibitory neurons in cortical model of 
neuronal networks with scale-free structure. With increasing 
F (triangles up, A) a discontinuous activation transition oc- 
curs at a critical noise level F c . With decreasing F (triangles 
down, v) the network remains in the metastable active state. 
N = 50000, 7 = 2.5, K ab = 100, Jj = -2.4, « = 18, a = 1, 
At = 0.1. 



B. Critical level of noise 

Our simulations revealed that the critical point F c of 
the discontinuous transition depends on both structure 
and size of the network. Our results are represented in 
Fig. [SI In networks with the degree exponent 2 < 7 < 3, 
such as scale-free networks revealed by fMRI in humans 
2], the critical noise level F C (N) decreases with increas- 
ing the size N following a power law, 



F C (N) oc N~ 



(18) 



The exponent a depends on the degree exponent 7: 
cr( 7 = 2.8) w 0.7 and cr( 7 = 2.2) w 0.82 (Fig. Hi). In 
the case of the Erdos-Renyi networks, the critical noise 
F c of the discontinuous activation transition is finite and 
does not depend on size at N ^> 1, see Fig. [5^,. The 
Erdos-Renyi networks were constructed using the meth- 
ods described in Section III Bl Neurons in the networks 
have the same mean degrees as the corresponding scale- 
free networks. 

In Appendix [Bl solving Eqs. (jlll) for a steady state, 
p e = pi — 0, we show that in the cortical model with 
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FIG. 6. Critical noise level F c of the first-order phase transi- 
tion versus network size N. (a) Scale-free neuronal networks 
(static model), (b) The same networks as in (o) but with 
shuffled in- and out-degrees. Each data point was obtained 
by averaging over 5 network realizations. Other parameters: 
gi = 0.2, K ab = 100, Ji = -3.0, f2 = 12, a = 1.0. 
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FIG. 7. Frequency v of sustained network oscillations versus 
the noise level F in a scale-free (SF) network (static model) 
and Erdos-Renyi (ER) network. Parameters in simulations: 
N = 50000, gi = 0.2, 7 = 2.5 (static model), K ab = 2000, 
Ji = -3.5, Q = 100, a = 0.1, At = 0.1. 

If the parameter a, Eq. (|8|). is smaller than a critical 
value a* defined in Sec. MI Al then with increasing the 
noise level F, the cortical model undergoes a phase tran- 
sition to a state with sustained network oscillations. In 
Fig. [71 we compare frequencies of sustained network os- 
cillations induced by noise in Erdos-Renyi and scale free 
neuronal networks having the same mean in- and out- 
degrees and constructed according to Sec. Ill Al and III Bl 
One can see that the frequency of the oscillations depends 
on both the noise level F and the network structure. Fur- 
thermore, network oscillations in the scale-free networks 
appear at a much weaker noise than in the randomly 
connected network. 



the scale- free degree distribution at 7 < 3, in the limit 
N — > 00, the critical value F c of noise in the activation 
process tends to zero, It means that the activation phase 
transition described in Sec. IIII Al takes place at F c = 0, 
i.e., at an arbitrary small noise level F a finite fraction 
of the neuronal network is activated. However, F c ^ if 
7 > 3 or N is finite. 

We would like to note that the absence of the threshold 
F c was obtained within the static model. In this model, 
in- and out degrees are correlated. If a neuron has a 
large in-degree, then with high probability it also has a 
large out-degree. If these correlations are removed, for 
example, shuffling in- and out-degrees like in papers (Til 
HTj], then, according to our simulations, a finite threshold 
is restored, though it is much smaller compared to the 
Erdos-Renyi networks, see Figs.|B^, and|BjD. 



IV. RANDOM DAMAGE AND TARGETED 
ATTACKS ON NEURONAL NETWORKS 

Alzheimer's disease is characterized by the loss of neu- 
rons and synapses in the cerebral cortex and certain sub- 
cortical regions and it is accompanied by the slowing- 
down of the alpha rhythm [30]. In this section, moti- 
vated by these observations, we study impact of random 
damage and targeted attacks on neuronal networks. 

A. Random damage of neuronal networks 

We start by investigating the impact of random re- 
moval of excitatory or inhibitory neurons on synchro- 
nization and frequency of sustained network oscillations 
in neuronal networks similar to scale-free networks re- 
vealed by fMRI in humans 0- In simulations, we stud- 
ied random damage of scale-free networks that consist 



8 



1.4 
1.2 
1.0 
0.8 
0.6 
0.4 
0.2 
0.0 

1.4 
1.2 
1.0 
0.8 
0.6 
0.4 
0.2 
0.0 



n e r = 
n e r =1200 
n e ;=2400 
n o (r, =3600 




200 



n i! r != o 

n: r =220 
n: r =440 
ni (r, =660 



(b) 



0.04 
V 0.02 
0.00 



250,500 750 

















■...-A ^ 







50 



100 
t 



150 



200 



FIG. 8. Sustained network oscillations of excitatory activity 
in randomly damaged scale-free networks, (a) Random re- 
moval of n^ r ' excitatory neurons, n^ r ' = 0, 1200, 2400, and 
3600. The corresponding fractions of randomly removed ex- 
citatory neurons are Q e = 0,0.15,0.3, and 0.45. (b) Removal 
of ' inhibitory neurons, n\ r) = 0, 220, 440, and 660. The 
corresponding fractions of randomly removed inhibitory neu- 
rons are Qi = 0,0.11,0.22, and 0.33. Insets: Frequency v 
of sustained network oscillations versus the number of re- 
moved excitatory or inhibitory neurons. Other parameters: 
N = 10000, g t = 0.2, 7 = 2.5, K ab = 75, F = 0.1, Jt = -3.5, 
ft = 10, a = 0.1, time step At = 0.1. 



of 10000 neurons (80% excitatory and 20% inhibitory 
neurons) and have a fat-tailed degree distribution with 
7 = 2.5 (the static model from Sec. Ill Aft . In Fig. [8j one 
can see that the sustained network oscillations are toler- 
ant to errors in sufficiently broad ranges of the fractions 
Q e and Qi of randomly removed excitatory or inhibitory 
neurons. With increasing Q e and Qi, the amplitude and 
frequency of sustained network oscillations decrease. Re- 
moval of about 56% of excitatory neurons or about 25% 
of inhibitory neurons suppresses completely the oscilla- 
tions. Thus, the network is more tolerant to random 
removal of excitatory neurons than to random removal 
of inhibitory neurons. The main mechanism of this effect 
is decrease of the network's connectivity that, in turn, 



attenuates synchronization between neurons. A similar 
decrease of power and frequency of alpha rhythm up to 
disappearance of alpha rhythms in the severe stages oc- 
curs in Alzheimer's disease [30l] . 

Above we found that scale-free neuronal networks dis- 
play a surprisingly high degree of tolerance against ran- 
dom failures. A similar effect occurs if, instead of neu- 
rons, a certain fraction of synapses is randomly removed. 
This assumption is based on the fact that, in complex 
netwo rks, bond and site percolation have similar prop- 
erties [16|. Robustness of scale- free neuronal networks 
to damage may be even enhanced if the networks have 
community structure 75| . 



Vulnerability of scale-free neuronal networks to 
targeted attacks 
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FIG. 9. Sustained oscillations of excitatory activity in scale- 
free neuronal networks damaged by targeted attacks, (a) Re- 
moval of ni h ^ excitatory hubs, (b) Removal of inhibitory 
hubs. Insets: Frequency v of the sustained oscillations versus 
the number of removed excitatory or inhibitory hubs. Pa- 
rameters: TV = 10000, gi = 0.2, 7 = 2.5, K ab = 75, F = 0.1, 
Ji = -3.5, Q = 10, a = 0.1, At = 0.1. 

Now we study the role of hubs in collective dynamics 
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of neuronal networks with scale-free structure. Using the 
same network with degree exponent 7 = 2.5 as above, 
we attack excitatory or inhibitory hubs and study sus- 
tained network oscillations. In the static model described 
in Sec. Ill Al if a neuron has a small index j then with a 
high probability it will have many connections, i.e., it will 
be a hub. Thus, removal of neurons one by one, start- 
ing from j = 1, corresponds to targeted attacks on hubs. 
Fig. [9] represents impact of these attacks on frequency of 
sustained network oscillations. One can see that removal 
of about 120 excitatory hubs or about only 30 inhibitory 
suppress completely sustained network oscillations. The 
observed vulnerability of oscillations to attacks is simi- 
lar to attack vulnerability found in Ref. [l3| in other real 
scale-free networks, such as the World Wide Web (www), 
Internet, social networks or a cell. The common proper- 
ties of these networks, including neuronal networks in 
the brain, is that their structure is characterized by a 
fat-tailed degree distribution with divergent second mo- 
ment. 



V. CONCLUSION 

In the present paper, we have studied the role of scale- 
free structure and the impact of noise and damage on 
dynamics of neuronal networks. We have developed an 
analytical approach and performed simulations of dy- 
namics of a cortical model with stochastic neurons and 
scale-free organization. We have found that, in scale-free 
networks with divergent second moment of degree dis- 
tribution (networks with degree exponent 2 < 7 < 3), 
such as functional networks in the human brain), a very 
small noise level can stimulate correlated activity of a 
finite fraction of neurons. The critical level of noise stim- 
ulating the activity decreases with increasing the net- 
work size and vanishes in the thermodynamic limit. The 
threshold level of noise, that is necessary to stimulate 
sustained network oscillations, is also strongly reduced 
in this kind of scale-free networks. Networks with a fi- 
nite second moment of degree distribution (scale- free net- 
works with 7 > 3) need a higher noise level to stimulate 
correlated neuronal activity and this level is finite in the 
thermodynamic limit. Furthermore, we have studied im- 
pact of random damage on sustained network oscillations 
in scale- free networks with 2 < 7 < 3. We demonstrated 
that, in these networks, network oscillations are toler- 
ant to random damage in a broad range of the number 
of randomly removed excitatory or inhibitory neurons. 
With increasing the number of removed neurons, the fre- 
quency of network oscillations gradually decreases similar 
to the slowing-down of the alpha rhythm in Alzheimer's 
disease. However, the networks are vulnerable to tar- 
geted attacks on hubs. A targeted attack on a few hubs 
can impair sustained neuronal oscillations in contrast to 
the case of random damage. Interestingly, the scale-free 
networks are more tolerant to removal of excitatory neu- 
rons than to removal of inhibitory neurons. Impact of 



damage is related to the fact that the removal of neu- 
rons decreases the network's connectivity and, in turn, 
attenuates synchronization between neurons. 
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Appendix A: Derivation of rate equations 

Here we derive the rate equations (jTT|l and (TT5]) for 
the cortical model with stochastic neurons and scale-free 
structure (the static model in Sec. Ill A[) . First, we find 
a rate equation for the probability /? a (^, t) that at time 
t neuron I in excitatory or inhibitory population, i.e., 
a = e or i, respectively, is active. For this purpose, we 
use the method of generating functions from paper [62| 
and generalize it to case of the cortical model with two 
neuronal populations in Sec. Ill Al The probability that 
presynaptic neurons I in population a is connected to 
postsynaptic neuron j in population b is p a ,e-,b.j, Eq. ([3]). 
For neuron j in population 6, we introduce a generating 
function, 



N a 



Gabij, x, y) = JJ j cxp {-p a ,i-fi,j) + 
£=1 

[xp a {l,t) + y(l - p a [l, t))](l - exp {-p a ,i-b,j))} « 

cxp I ~Y] p a ,i- b,j + [xp a (£, t)+y(l-p a (£, i))]p ,4&,j ■ }> 

(Al) 



1=1 



1=1 



where we used the fact that p a ,£-.b,j <C 1 at N 3> 1. One 
can show that the probability that neuron j in population 
b has n active presynaptic neurons in population a at time 
t is equal to 



1 d n 
n\ dx 1 



■G ab (j,x,y) 



x—0,y—l 



(A2) 



The condition y = 1 means that other presynaptic neu- 
rons are inactive. Using Eqs. (|T4")) and (ITU1) for C a b{j) and 
the weighted activity p a (t), we obtain that the probabil- 
ity (|A2|) is equal to P n (Pa{t)C a b{j))- Note that this prob- 
ability depends on the weighted activity p a , Eq. (fTTJjl . of 
presynaptic neurons rather than the activity p a , Eq. 
that determines the fraction of active neurons in pop- 
ulation a. This is in contrast to the cortical model on 
sparsely connected network [5lT | where the probability de- 
pends on p a . 
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Using the probability (IA2[) . we can find the probability 
that neuron j in population b has n active presynaptic 
excitatory and m active presynaptic inhibitory neurons. 
It is equal to P n (p e (t)C eb (j))P m (pi(t)Ci b (j)). This prob- 
ability allows us to calculate the probability ^b,j{f>e, Pi) 
that an input at neuron j in population b at time t is at 
least the threshold fi, see Eq. (flU)) . 

Based on the dynamical rules in Sec. (|II Cj) and the 
method of Ref. [51| , we find a change of the probability 
Pb{j,t) during the integration time r, 

Pb(j, t + T) - p b (j, t) = r/6 [1 - Pb(j, t)] + 
Tfl b [l-p b (j,t)]^ b j(p e , pi)-Tflbpb(j,t)[l-^b,j(pe, Pi)]- 

(A3) 

The first two terms on the right-hand side describe acti- 
vation of neuron j by noise and input from presynaptic 
neurons with the probabilities rfb and r/Xb, respectively. 
The last term describes decay of p b (j,t) with the proba- 
bility r/Zb, if input is smaller than the threshold f2. As- 
suming that characteristic time scales of the collective 
dynamics are much larger than the integration time r, 
we obtain a rate equation, 

Pb(j, t) = fb - VbPb{], t) + p b V b j(p e , Pi). (A4) 

where v b = f b + p, b - Multiplying this equation by the 
weight w b (j) or 1/N b and summing over j, we obtain 
Eqs. (ITT1) and (fTo)) . respectively. 



Appendix B: Absence of an activation threshold in 
the static model 













' ■ / „ Pe 




. .. V ? e (Pe. 7^2.2) - " . 




?e(Pe. T=2.6) 




0' Te(Pe. T=3.0) 




^(^,7=3.4) 




V e (p e , 7=3.8) 




•I'oiPo- 7 "-2) 



0.2 0.4 0.6 0.8 1 

Pe 



FIG. 10. Function * e (p e ,p"i) denned by Eq. (fT2"1) versus p c = 
Pi at different values of the degree exponent 7. N = 10000, 
g % = 0.2, Kab = 100, Ji =-2,Q= 18. 



In the steady state, the rate equations (|TT1) take a form, 

p e = F e + (l-F e )V e (p e ,pi), 

Pi = F i + {l-F i )^ i (p e ,p i ), (Bl) 

where F e and Fi are defined by Eq. (|17|) . Solving non- 
linear Eqs. (|B1|) . one finds a dependence of weighted neu- 
ronal activities p e and pi on the noise level F, where 
F = F e = Fi. From Eq. (|T2|) . we find that in the case 
2 < 7 < 3, the functions ^> e (p e ,pi) and ^i(p e ,pi) have a 
singular asymptotic behavior at small activity p e ,Pi -C 1, 

^ a (p e ,p,)«^pr 2 - (B2) 

Therefore, <N> a /dp e — > 00 at p e — > 0. One can see this 
singular behavior in Fig. [10] that displays the function 
ty e (p e ,Pi) versus p e at p e = p^ at different degree expo- 
nents 7. Substituting Eq. (IB2|) into Eqs. (jBll) . one finds 
that, in the case 2 < 7 < 3, the only solution of Eqs. (|Blj) 
is a solution with p e and pi of order O(l) at any non-zero 
F. However, at 7 > 3, the function ^ e (p e ,pi) does not 
have this kind of singularity and (N> a /dp e —¥ at p e — > 0, 
see Fig.[l0l In this case, Eqs. (jBlj) have a stable solution 
p e k p l k. F at F < 1. 

Appendix C: Activation rate of neurons stimulated 
by synaptic noise 

In the cortex, the observed mean rate wi? of spon- 
taneous release of neurotransmitters in a synapsis is in 
the range [0.01 Hz, 4 Hz]. This is the source of synaptic 
noise. If an excitatory or inhibitory neuron has q presy- 
naptic connections, then the rate of random inputs is 
uj sn = q win ■ If there are no correlations between spon- 
taneous release of neurotransmitters in the synapses and 
the process is described by a Poisson process, we find that 
the probability that during the integration time r a neu- 
ron receives n random inputs is P„(ro; sn ) where P n {x) is 
the Poisson distribution function. The probability that 
a neuron receives at least spikes during time r is 

P act = Pn{TU S n). (CI) 

For simplicity, in these calculations, we neglected a spon- 
taneous release of neurotransmitters in synaptic connec- 
tion with inhibitory presynaptic neurons and took into 
account only connections with excitatory neurons. If the 
mean number of presynaptic connections is q w 1000, the 
integration time r is 10ms, the threshold 51 = 30, and the 
rate win in the interval 1-4 Hz, then we obtain that the 
probability P act varies from 2 x 10~ 7 to 0.96. In turn, the 
probability P ac t is related with a rate / of the activation 
process, P ac t = fr. This relationship determines the rate 
/ of activation of neurons by synaptic noise. 
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